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DETAILED ACTION 

This application has been examined. 

Election/Restrictions 
Restriction to one of the following inventions is required under 35 U.S.C. 121: 

I. Claims 1-13 23-24, drawn to Group 1, classified in class 345, subclass 
752. 

II. Claims 14-22, drawn to Group 2, classified in class 345, subclass 716. 

1 . The inventions are distinct each from the other because of the following reasons: 
Inventions I and II are related as subcombinations disclosed as usable together in a 
single combination. The subcombinations are distinct from each other if they are shown 
to be separately usable. In the instant case, invention I has a different mode of 
operation to provide a electronic mail system with enhanced capabilities of attaching 
audio and video data to electronic file. In the instant case, invention II has a different 
mode of operation transmitting data packets via network by concatenating audio, video, 
and text data packets into a single packet. See MPEP 806.05(d). 

2. Because the inventions are distinct for the reasons given above and have 
acquired a separate status in the art because of their recognized divergent subject 
matter, and because the search required for Group I is not required for Group II a 
restriction for examination purposes is indicated is proper. 

3. During a telephone conversation with Applicants' Attorney, Robert Holland, on 
September 21 , 2004, Group 1 elections was made. 
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4. Applicant is reminded that upon cancellation of claims to a non-elected invention, 
the inventorship must be amended in compliance with 37 CFR 1 .48(b) if one or more of 
the currently name inventors is no longer an inventor of at least once claim remaining in 
the application. Any amendment of inventorship must be accompanied by a request 
under 37 CFR 1.48(b) and by the fee required under 37 CFR 1.1 7(i). 

Applicant is advised that the reply to this requirement to be complete must include an 
election of the invention to be examined even though the requirement be traversed (37 
CFR 1.143). 

Drawings 

5. The Examiner contends that the drawings submitted on July 05, 2000 are 
acceptable for the examination proceedings. 

Claim Rejections - 35 USC § 102 
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 
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6. Claims 1-4,6-13, 23-24 are rejected under 35 U.S.C. 102(e) as being anticipated 
by Cleron et al (U.S. Patent Number 6,223,213). 

7. In regards to claim 1 , Cleron discloses a method of asynchronous 
communication by a plurality of users via a network of interconnected computers, 
comprising: 

selecting, at a client site, a file and capturing said file as an image file(col. 2 lines 
1-3) [The thin client is equipped with a microphone and video 
input to receive audio and video data .]. The examiner interprets that 
client has the ability to receive audio and video data from the client's microphone and 
video input; 

creating, at said client site, an audiovisual message, said audiovisual message 
relating to said tile, said creating comprising capturing digital information (col. 2 lines 1- 
3) [The thin client is equipped with a microphone and video input 
to receive audio and video data .]. The examiner interprets that by the client 
equipped with a microphone and video input, the purpose of this items are create and 
capture digital information for an audiovisual message; 

appending said image file and said audiovisiJarmessage^to achieve a combined 
message (col. 1 line 66 - col. 2 line 1) [the email system allows the thin 
client to capture audio and video data for inclusion with the 
email message .]. The examiner interprets that by capture the audio and video data 
the electronic mail being sent out is also an audiovisual message; and 
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electronically delivering said combined message to at least one of said plurality 
of users (col. 2 lines 18-20) [After the user has reviewed the email 
message, the user can click a "Send" link on the rendered email 
page to- send the email message to an intended recipient.]. 

8. In regards to claim 2, Cleron discloses said electronically delivering comprises 
streaming said digital information via said network (col. 2 lines 24-27) [the host mail 
server converts the email message and the audio or video data to 
a MIME message (Multipurpose Internet Mail Extensions) and 
forwards the MIME message to the intended recipient, where it 
can be rendered in full .]. The examiner interprets that a mail server is used to 
electronically send data across a network. 

9. In regards to claim 3, Cleron discloses further comprising creating a reply record, 
said reply record having audiovisual material, said reply record being in response to 
said combined message (col. 2 lines 3-6 & col. 1 line 66- col. 2 line 1) [The c lient 
br owser supports a user interface that includes pop-up capture 
panels for both audio and video, with each capture panel 
enabling the user to record a select~ed""c~lip~~and add the clip to 
the email message .] [the email system allows the thin client to 
capture audio and video data for inclusion with the email 
message .]. The examiner interprets that user has the ability to create audiovideo 
message, so if a user were to receive an email, the user has the ability to create a reply 
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using the same feature to create an audiovideo message to reply in response to the 
email. 

10. In regards to claim 4, Cleron discloses said reply record is created by one of said 
plurality of users, and further wherein said one of said plurality of users annotates said 
image file (col. 1 line 66- col. 2 line 3) [According to one aspect of the 
invention, the email system allows the thin client to capture 
audio and video data for inclusion with the email message. The 
thin client is equipped with a microphone and video input to 
receive audio and video data.]. The examiner interprets the claim for creating 
a reply that has both audio and video data file, with in the combined electronic mail. 

11. In regards to claim 6, Cleron discloses said creating comprises accessing a user 
interface, said user interface having stateful buttons (col. 2 lines 3-6) [The client 
browser supports a user interface that includes pop-up capture 
panels for both audio and video, with each capture panel 
enabling the user to record a selected clip and add the clip to 
the email message.]. 

12. In regards to claim 7, Cleron discloses said selecting further comprises activating 
said image file for editing (col. 2 lines 18-20) [After the user has reviewed the 
email message, the user can click a "Send" link on the rendered 
email page to send the email message to an intended recipient.]. 
The examiner interprets that the user has the ability to review the electronic mail before 
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it is being sent is a form of editing, because if the user does not like the recording for 
audio or video file, the user has the ability to recapture the files. 

13. In regards to claim 8, Cleron discloses said electronically delivering comprises 
transmitting said combined message to a central server for storage and further 
transmission (col. 2 lines 24-27) [Upon receiving this second request, the 
host mail server converts the email message and the audio or 
video data to a MIME message (Multipurpose Internet Mail 
Extensions) and forwards the MIME message to the intended 
recipient, where it can be rendered in full.]. 

14. In regards to claim 9, Cleron discloses a method of creating and delivering an 
electronic audiovisual message via a network of computing means, comprising: 

capturing at least a portion of an electronic document as an image file (col. 2 
lines 1-3) [The thin client is equipped with a microphone and video 
input to receive audio and video data .]. The examiner interprets that client 
has the ability to receive audio and video data from the client's microphone and video 
input; 

annotating said image file (col. 1 line 66 - col. 2 line 1) [the email system 
allows the thin client to capture audio and video data for 
inclusion with the email message.]; 

creating said electronic audiovisual message, said electronic audiovisual 
message regarding said electronic document (col. 2 lines 1-3) [The thin clie nt is 
equipped with a microphone and video input to receive audio and 
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video data .]. The examiner interprets that by the client equipped with a microphone 
and video input, the purpose of this items are create and capture digital information for 
an audiovisual message; 

combining said annotated image file and said electronic audiovisual message 
into a combined message (col. 1 line 66 - col. 2 line 1 ) [the email system allows 
the thin client to capture audio and video data for inclusion 
with the email message .]. The examiner interprets that by capture the audio and 
video data the electronic mail being sent out is also an audiovisual message; and 

delivering said combined message to at least one desired recipient via said 
network(col. 2 lines 24-27) [Upon receiving this second request, the host 
mail server converts the email message and the audio or video 
data to a MIME message (Multipurpose Internet Mail Extensions) 
and forwards the MIME message to the intended recipient, where 
it can be rendered in full.]. 

15. In regards to claim 10, Cleron discloses replying to said combined message via 
said network, wherein said replying comprises creating an audiovisual reply designated 
for insertion at a predetermined point in said combined message (Figure 8). 

16. In regards to claim 1 1 ; Cleron discloses said replying comprises delivering said 
audiovisual reply with an address pointer indicative of said predetermined point (Figure 
8). The examiner interprets that in Figure 8 the user has the replay electronic mail with 
audiovisual reply to address where it should be delivered. 
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17. In regards to claim 12, Cleron discloses said delivering comprises storing said 
combined message at a remote location and sending an electronic notification having a 
uniform resource locator corresponding to said location (col. lines 47-50). The email 
system uses HTTP, which also uses uniform resource locator. 

18. In regards to claim 13, Cleron discloses said delivering comprises streaming said 
combined message to said at least one desired recipient via the Internet (col. 2 lines 24- 

27) [Upon receiving this second request, the host mail server 
converts the email message and the audio or video data to a MIME 
message (Multipurpose Internet Mail Extensions) and forwards the 
MIME message to the intended recipient, where it can be rendered 
in full.]. 

19. In regards to claim 23, Cleron discloses a machine-readable storage medium 
comprising a set of instructions executable by a computer system to implement a 
method, the method comprising: 

providing a graphical user interface to allow a user to select desired content as 
an image file; 

permitting said user to annotate said image file (Figure 5; 
capturing audiovisual material responsive to said image file as a digital file 
(Figure 6 & Figure 7); 

pairing said digital file and said annotated image file as a combined file (Figure 

8); and 
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delivering said combined file to at least one desired location via said computer 
system (col. 2 lines 24-27) [Upon receiving this second request, the host 
mail server converts the email message and the audio or video 
data to a MIME message (Multipurpose Internet Mail Extensions) 
and forwards the MIME message to the intended recipient, where 
it can be rendered in full .]. The mail server is considered to be a computer. 
20 . In regards to claim 24, Cleron discloses wherein the machine-readable storage 
medium includes any of magnetic storage medium, including disk and tape storage 
medium 1 , optical storage medium, including compact disk memory and digital video disk 
storage medium; nonvolatile memory storage memory; volatile storage medium; and 
modulated, electronic signals, (col. 9 line 45- col. 10-17) [13 . a thin client 
comprising: a processor; a memory coupled to the processor; a 
data input to receive a data stream; a browser program stored in 
the memory and executable on the processor, the browser 
rendering pages written in a markup language to present an email 
interface for reading and writing email messages; a user 
interface stored in the memory and executable on the processor 
to enable capture of the data stream received at the data input 
for inclusion in an email message; wherein upon a user command 
to add the data stream to the email message, the 
browser submits a request to a host mail server for a second 
page that contains the email message with the data stream 
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included therein; and wherein the browser receives a response 
specifying a new page that contains 

the email message and indicating a location in the second page 
at which the data stream is to be rendered.] 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1 , 148 

USPQ 459 (1966), that are applied for establishing a background for determining 

obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Cleron et 
al (U.S Patent Number 6,223,213) in view of Freivald (U.S. Patent Number 6,219,818). 

Cleron shows a browser-based email system that incorporates audio and video 
data with the document but does not show annotating at least a portion of said image 
file using a cursor-based highlighter. 

Freivald shows annotating at least a portion of said image file using a cursor- 
based highlighter (col. 7 lines 10-13) [The user can select paragraphs of 
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text by dragging a highlight across the text . Responder 2 4 then 
stores the location of the selected text and generates one or 
more CRC for the selected text.] 

It would have been obvious to a person of ordinary skill in the art at the time of 

the invention was made to combine Cleron, browser-based email system, with Freivald, 

hightlighting text document. 

The modifications would have been obvious because one of ordinary skill in the 

art would have been motivated to combine because it allows the user to send electronic 

mails that contain audio, video, and text, to aid the user in identify relevant portions of 

the electronic mail document. 



Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to O'Neal R Mistry whose telephone number is (703) 305- 
2738. The examiner can normally be reached on 9am - 6pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John W Cabeca can be reached on (703)308-31 16. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 

O'Neal Mistry 

Assistant Patent Examiner 

Art Unit 2173 

o'neal. mistry@uspto.gov 
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